1. Field of the Invention
The present invention relates generally to a handwriting information processing technique, and more particularly, the present invention relates to a handwriting information processing technique for allowing the user to input handwritten characters and convert them into typed text at a desired timing so as to be displayed on a display apparatus.
2. Description of the Related Art
Examples of a handwriting information processing apparatus having means for recognizing handwriting in the related art are described below.
(1) In Japanese Patent Laid-Open No. 2000-099223, a data processor (handwriting information processing apparatus) having an interface with fixed handwriting input spaces wherein recognition errors can be easily corrected is disclosed. In this data processor, a character code corresponding to the handwriting is sent to the relevant application.
(2) In Japanese Patent Laid-Open No. 2001-014103, a character input device (handwriting information processing apparatus) having an interface with fixed handwriting input spaces, as in the invention of (1), wherein an input method can be switched in an efficient manner is disclosed.
(3) In Japanese Patent Laid-Open No. 2000-076380, a handwriting input device (handwriting information processing apparatus) that allows inputting of the handwritten characters in any direction without having to consider fixed input spaces or designated text format upon inputting handwritten characters on a touch panel is disclosed.
(4) In Japanese Patent Laid-Open No. 09-319503, a text processing apparatus (handwriting information processing apparatus) that can be easily manipulated to perform text inputting and text editing using a combination of a written character string input method wherein the handwritten characters do not go through a recognition process and are handled as stroke (a plurality of points connecting one end to the other in the, handwritten character) data, and a recognized character string input method wherein the handwritten characters are recognized and converted into character code is disclosed.
(5) In Japanese Patent Laid-Open 2000-043485, an electronic blackboard (handwriting information processing apparatus) for use in a conference and the like, wherein handwritten characters input to a touch panel are processed for character recognition, converted into typed text, and displayed on the touch panel is disclosed. According to this invention, neat typed text can be obtained in place of a sloppy handwritten version of the topic and minutes of a conference, for example.
The above descriptions of the related art are samples of prior inventions relating to handwritten character recognition apparatuses, particularly, to the user interface of the apparatus. The handwritten character recognition user interface can be roughly divided into two types. One type is for the likes of inventions (1) and (2) wherein fixed input spaces are provided and the user is required to input one character in each space. This type is the most conventional user interface. In this type of user interface, it is unnecessary to perform a cut-out process for each of the characters and recognition errors due to an error in the cut-out process can be avoided. However, this type of interface imposes a great burden on the user and is far from being user-friendly (hereinafter, handwritten character recognition in such a type of user interface is referred to as ‘fixed space handwritten character recognition’). The other type of user interface includes the above inventions (3) and (4), wherein no fixed input spaces are provided so that the user can write freely into the handwriting input region, and a cut-out process for each of the characters is performed in order to recognize the handwritten characters. In this type of user interface, recognition error does occur as a result of the character cut-out process; however, this type of interface is advantageous in that the user can input characters with the ease of writing with a conventional pencil (hereinafter, the handwritten character recognition in such a type of user interface is referred to as ‘free handwritten character recognition’).
The conventional methods of handwritten character recognition are the above-described fixed space handwritten character recognition and the free handwritten character recognition. In these conventional recognition methods, although conversion into typed text is performed based on information on the shapes of the hand written characters, no process is undertaken concerning the position at which the characters are written nor the size thereof. Thus, in the conventional method, the handwritten characters input by the user are displayed at a predetermined character position and converted into a code character of the same size regardless of the size or the position in which the original handwritten characters have been input. For example, in a handwritten text of the minutes of a conference and the like, the user may want to emphasize a word/phrase, to enlarge the characters of an important word/phrase, or leave a blank to be filled in later In the conventional art, it is impossible to perform all of these operations in the initial text conversion process. Thereby, the user has to perform individual processes after the text conversion process for changing the font size of a particular word/phrase or moving the existing characters by inputting spaces in the desired area to add a word/phrase.
Since performing such operations imposes a great burden on the user, it is preferable that the conversion process be simpler. Thus, a handwriting information processing apparatus with a user interface that is capable of reflecting the intentions of the user inputting the handwritten characters, such as displaying typed characters in a large font for the corresponding handwritten characters that should be emphasized or providing a blank space between characters.
Also, when the user is able to write freely into the input region of the handwriting information processing apparatus, it is difficult to realize 100% character recognition, that is, a 100% probability of the handwritten characters being converted to the code characters originally intended by the user, due to the handwriting peculiarities of each user. Thus, it is necessary to edit the code characters displayed in response to the handwritten character input. This also imposes a burden on the user, and therefore, it is preferable that very simple operations enable various editing processes. Also, editing operations that are intuitive in nature and easily associated or unexpected and amusing are more likely to be accepted by the user since the learning of these various editing operations can be facilitated.
In a conference and the like, it is imperative that important subjects and propositions that are discussed be organized in well-devised minutes so that all members of the conference are able to comprehend the discussion in a restricted time period. Thus, in the user interface of a handwriting information processing apparatus used in a conference such as an electronic blackboard, operations such as emphasizing a heading, adding a postscript, and the like should be made simple for the user.
Further in the conventional art, recognition processing is performed for each character or for a string of characters accumulated until a certain length is reached, after which the recognized character string is shown and the user determines the desired character (changes to the character string). This method of progressively converting the handwritten characters is not very suitable for use in a conference. That is, the necessity of checking the recognition results of the handwritten characters right in the middle of a discussion may arise, which may result in an interruption in the flow of thoughts. Thus, in converting handwritten information during a conference into type text, the handwritten information is preferably saved as coordinate values, wherein the actual conversion of the information into typed text is completed when the discussion sinners down or at the end of the conference.
Additionally, in a conference, the positioning of the written character strings is quite important and a character string usually has a meaningful relationship with the character strings written above and below it. Thus, it is preferable that the outline of the character strings that are represented as a list or in an indent format, for example, be represented as they are written. Further, if the outlined character strings can be easily saved in a file or sent to another person, the information generated from the conference can be put to effective use.
Also, in the conventional method, when a user wants to convert handwritten characters input as ‘hiragana’ characters into Chinese characters, the user has to take an action such as pushing a Chinese character conversion button and the like. In such a case, the IME (Input Method Editor) by Windows (registered trademark) performs Chinese character conversion only for the ‘hiragana’ character portion of the character string even when it contains both ‘hiragana’ and Chinese characters, thereby leaving the portion originally handwritten in Chinese characters unconverted. Thus, it is preferable that the Chinese character conversion and the handwriting information processing into typed text be performed simultaneously.